Adding regularization objective option to parmest#3550
Adding regularization objective option to parmest#3550sscini wants to merge 40 commits intoPyomo:mainfrom
Conversation
|
@djlaky @adowling2 Please provide early feedback |
sscini
changed the title
pyomo/contrib/parmest/parmest.py
Outdated
|
|
||
| Added to SSE objective function | ||
| """ | ||
| expr = ((theta - theta_ref).transpose() * prior_FIM * (theta - theta_ref) for theta in model.unknown_parameters.items()) |
There was a problem hiding this comment.
Does this run? My intuition is that you need to write out the matrix multiplication and cannot use matrix multiplication.
|
UPDATE (07/15/25); Intended to be added AFTER Shammah's weighted SSE PR (#3535) and follow a similar format, adding another option for available objectives using Enums. Currently preparing for next stage of review |
|
@adowling2 @djlaky Made some edits to this finally. Meant to follow Shammah's layout in PR #3535, waiting to merge, currently some conflicts I cannot resolve on my end. Made it's own full objective instead of a term to add. Also added weighting term. Ready for next round of review. |
sscini
changed the title
blnicho
changed the title
|
Will not be actively worked on before finalization of _Q_opt_blocks. Closing for now to resume in the future. |
|
Got L2 regularization working, still need to modify to improve implementation slightly. Made prior_FIM and theta_ref as pd.DataFrames because this is how parmest would return them, and would not be reliant on assuming order is correct. |
|
Made theta_ref a pd.Series so if you take theta and invert the covariance outputs from theta_est(), you can feed them directly back in. |
|
@adowling2 L2 functionality working. Made theta_ref a pd.Series and prior_FIM a pd.DataFrame so parmest outputs can be used as inputs. All previous tests passing. Added preliminary toy example using rooney_biegler to show movement of estimated theta and reduction of covariance. Please review when available. |
sscini
moved this from Development
to Ready for design review
in ParmEst & Pyomo.DoE Development
|
@slilonfe5 @adowling2 @blnicho @mrmundt Please review at your earliest convenience. Questions are included within the added code as comments. Please give preference on implementing the addition to the calculated covariance. Thanks! |
|
After conversation with Alex, made implementation decision. Method is applied in compute_covariance_matrix(), before FIM is inverted, preventing numerical issues from need to invert final covariance, add prior_FIM, and invert again if applied at end of _cov_at_theta. Currently supported for default (finite_difference) and automatic_differentiation_kaug methods. Need to find alternative implementation for support with reduced_hessian method. |
Weight was not symbolic, so was not affecting parameter estimation. Made significant changes to adjust structure so the regularization weight is calculated within the Estimator class, and applied to the covariance update as well.
3 participants
Fixes # .
Summary/Motivation:
Currently, the only default objective is the standard SSE objective. This edit provides the capability to add a
regularization term to the SSE objective with a prior FIM and reference parameter values.
Changes proposed in this PR:
TODO before converting from draft
Legal Acknowledgement
By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution: